Video information producing device

ABSTRACT

An video information producing device comprising an input device for inputting 3-D image information of a real image, and a processing device for processing 3-D image information from the input device, the processing device having a function of storing real-image 3-D image information and a function of giving motions to real-image 3-D image information read from the storing function by applying an image processing technique to generate moving image information.

TECHNICAL FIELD

The present invention relates to a video information producing devicefor producing a moving image by the computer graphics technology.

BACKGROUND ART

A video information producing device for producing a moving image by thecomputer graphics (CG) technology finds wide applications in the movieindustry and the commercial message sector. Especially in the movieindustry, the time has already come when the CG technology isindispensable for the most advanced movies in which animals created bythe CG technology play a principal role and men grow into robots.

Technological elements of the CG include the creation of athree-dimensional object (called modeling), taking a picture of athree-dimensional object by radiating light on a particular object(called rendering) and the creation of a moving image (calledanimation).

Modeling includes detailed techniques such as texture mapping fordressing a three-dimensional object and furring for attaching fur of ananimal.

The modeling work is to observe a three-dimensional object beingrendered and correct any nonconforming points. For an animal in motion,a skeleton is determined and an image is produced with fixed joints andmovable bones moved little by little. The resulting image is stored toproduce a screen for objective movement. In order to change complicatedmovements into natural ones, a measuring instrument for measuring theposition of the joints is worn by a man in motion thereby to obtain thedata on the movement of the joints (motion capture step).

In the case of a comic, parts concerning the moving image informationare prepared by the input work. It is a feature of CG that even a worldnot actually existent can be expressed.

The foregoing is an outline of the CG technologies explained in such areference as “Fundamental Knowledge of Computer Graphics”, by AtsushiShiokawa, published by Ohm. The CG technologies thus have already foundpractical applications.

Parts for making up a moving image consist of movable parts andimmovable background parts. In the conventional CG technology forproducing a moving image, both the movable parts and the backgroundparts are prepared manually by operators as three-dimensional imageinformation using the computer. This job requires a high-speed computerand a great amount of labor in view of the need of producing movingimages one by one.

Also, with regard to real objects such as actual men, animals, trees,houses, mountains, rivers and commodities, the CG technology is still inits infant stage as the three-dimensional image information thereof arestill handled as a moving part or a background part. In other words, thetechnologies for handling real objects as freely movable parts have yetto be completed.

Accordingly, the primary object of the invention is to provide a videoinformation producing device which can be used with a low-speed imageinformation production device such as an inexpensive personal computerto produce CG moving images and especially to handle thethree-dimensional image information on real objects as a moving part ora background part, thereby making it possible to develop the CGtechnologies further while at the same time reducing the cost of programproduction.

DISCLOSURE OF THE INVENTION

(1) With the video information producing device according to thisinvention, the object described above can be achieved by introducing thefollowing functions. Prior to the explanation of the functions, theterms used herein will be described. The real image is defined as animage of a real object which has an infinite variety of types.

A moving image can be divided into moving objects, immovable, stationaryobjects and a background. The immovable stationary objects can begrouped into backgrounds. Images of the moving objects constituting realparts and backgrounds photographed or imaged from a plurality ofdirections at the same time are real-image three-dimensional imageinformation, which are called moving parts and background parts,respectively. These objects can be handled by the image processingtechnology such as the computer graphics technology.

The video information producing device according to the invention usesan input device for the real-image three-dimensional image informationmaking up a real part. The input device includes, for example, ano-contact three-dimensional digitizer and a three-dimensional lasershape measuring system.

These devices can capture the three-dimensional image information of areal part (object) into a processing device such as a computer. As analternative, the three-dimensional image information can be producedfrom the photographic information taken from a plurality of angles of areal part.

This real-image three-dimensional image information can be inputautomatically, and therefore working labor can be saved. In the casewhere a background is really existent, the photographic imaging from aplurality of angles is all that is required, thereby reducing productioncost of large props and location cost. The computer making up aprocessing device for inputting the real-image three-dimensional imageinformation is generally a high-speed computer.

Also, the video information producing device according to the inventioncomprises a processing device for processing the real-imagethree-dimensional image information input by the input device describedabove.

The processing device has a storage function and a function ofgenerating moving image information by attaching motions to thereal-image three-dimensional image information read from the storagefunction using the computer graphics technology. This processing devicewith the moving image information generating function reads thereal-image three-dimensional image information from the storagefunction, and attaches the desired motions to the real-imagethree-dimensional image information using the required image processingtechnology such as the computer graphics technology thereby to generatemoving image information automatically.

A plurality of screens making up a moving image can be calculated andplotted by the processing device by handling a real image asthree-dimensional image information. Therefore, the screens are notrequired to be plotted one by one, thereby saving a great amount oflabor.

This makes it possible to automatically generate an image from variousviewpoints based on the three-dimensional image information of parts bythe processing device. The reason is that in spite of the movement ofparts, all the images of a moving image configured of a plurality ofscreens indicating the movement can be generated from the samethree-dimensional image information.

As described above, according to this invention, a moving image of areal object or a real person can be easily produced automatically usingthe computer graphics technology constituting an image processingtechnique. As a result, the following advantages can be obtained.

Take an entertainer as an example of a real man. Once the real-imagethree-dimensional image information of the entertainer is acquired inadvance instead of causing the entertainer to make a performance, thereal-image three-dimensional image information of the entertainer can beread at an arbitrary date and time and can be moved freely within arange of the image processing technique thereby to prepare the movingimage information of required motions even in his/her absence. Actualperformance of the entertainer is thus not required, so that his/herbinding time can be reduced considerably for a lower production cost.

It is practically impossible for an actual entertainer to perform such adangerous action as diving from a high place or such an unrealisticaction as jumping over a wide river. In the case of the moving imageinformation produced using the image processing technique based on thereal-image three-dimensional image information of the entertainer,however, such a restriction is not imposed, so that the moving imageinformation can be freely produced in any desired situation withinlimits of the computer graphics technology.

With regard to the background, the three-dimensional image informationon an actual object is used and therefore large props are not required,thereby making possible considerable reduction in program productioncost.

As described above, a video information producing device according tothe invention comprises an input device for inputting real-imagethree-dimensional image information and a processing device such as acomputer for processing the three-dimensional image information from theinput device, wherein the processing device includes a storage functionfor storing the real-image three-dimensional image information inputfrom the input device and a function of generating moving imageinformation by attaching motions to the real-image three-dimensionalimage information read from the storage function using an imageprocessing technique such as a computer graphics technique.

With this configuration, the real-image three-dimensional imageinformation making up a real component part can be automatically inputand therefore working labor can be reduced.

Also, desired motions can be added by a required computer graphicstechnique to the real-image three-dimensional image information readfrom the storage function thereby to generate the moving imageinformation automatically.

With regard to real images, it is handled as three-dimensional imageinformation and a plurality of screens making up a moving image can becalculated and plotted by a computer. For this reason, the screens arenot required to be plotted one by one, and therefore a great amount oflabor can be saved. With regard to backgrounds, on the other hand, theneed of large props is eliminated, again contributing to considerablereduction in the program production cost.

(2) A video information producing device according to the inventioncomprises an input device for inputting real-image three-dimensionalimage information and a processing device for processing thethree-dimensional image information from the input device, wherein theprocessing device includes an input function for inputting thereal-image three-dimensional image information, a storage function forstoring the real-image three-dimensional image information from theinput device, a small image information generating function forgenerating small image information from the real-image three-dimensionalimage information read from the storage function, a storage function forstoring the real-image three-dimensional image information and the smallimage information as associated with each other, a function ofdisplaying a scene frame for three-dimensional space expression, afunction of reading the small image information from the storagefunction and displaying the small image in the small image frame, afunction of selecting the small image information in the small imageframe on a screen and attaching by moving it to a desired position inthe scene frame, the function of storing the information for arrangingthe attached small image in the scene frame in the scene frame, afunction of correcting in the scene frame the mode of expression of thesmall image displayed within the scene frame, a function of changing inthe scene frame the mode of operation of the small image displayed inthe scene frame, a function of updating arrangement information of thesmall image after correction or change, a moving image informationproducing function for producing a series of temporal moving imageinformation by calculating the motion change from the small image beforechange to the small image after change in the scene frame, and afunction of storing the moving image information produced.

This invention deals with two types of three-dimensional imageinformation of a part including detailed image information and simpleimage information.

The three-dimensional image information of a part is configured of theinformation on the position and color at a point on the surface of thepart. Nevertheless, the color information may lack (monochromatic).

The detailed image information is described with a high density ofsurface points, while the simple image information has a surface-pointdensity about one hundredth (one example) smaller and so is theinformation amount. Incidentally, these numerical values are only anexample for reference and may assume an arbitrary value.

Although an image produced from the simple image information is rough,it should be possible to recognize a moving image. A screen is called asmall image of a part in the case where an image as viewed from thefront of the part (an example) is produced from the simple imageinformation or a comic (an example) indicating the particular part isproduced, and a number or a name unique to the part is attached to thescreen. This small image is assumed to contain the informationassociated with the detailed image information or the simple imageinformation of the part.

In a computer graphics production system having functions of modeling,ditigizing, rendering, etc. for example, small image informationgenerated from real-image three-dimensional image information isdisplayed in a small image frame (called thumb nail display).

Normally, a plurality of pieces of small image information aredisplayed. A small image of the small image information displayed in thesmall image frame is selected by, say, the drag-and-drop operation andattached by being moved to a desired position in the scene frame. In theprocess, the position where the small image is attached is stored asarrangement information.

This job is repeated to form a provisional scene composed of acombination of a background and a moving object in the scene frame. Theform of expression of the small image in the scene frame can becorrected to produce a better scene.

The correction includes fine adjustment of size, rotation, perspectiveand position upward, downward, leftward and rightward. Also, the form ofmotions of the small image can be changed. The form of motions includes“stand”, “sit down”, “lie down”, “jump”, “walk”, “run”, etc. With thecorrection or change, the arrangement information of the small imageafter particular correction or change, as the case may be, is updated.

In order to attach motions to a small image, the moving imageinformation producing function is used to calculate the motion changefrom the small image before change to the small image after change inthe scene frame and produce a temporal series of moving imageinformation. The moving image information thus produced is stored.

In the foregoing described operation, a computer can calculate and plota plurality of screens making up a moving image with the small imageassociated with the real-image three-dimensional image information as areally existing part. Thus, the screens need not be drawn one by onethereby to save the labor.

This is because an image can be generated automatically from variousview points based on the three-dimensional image information of a partand even in the case where the part moves, all the images of the movingimage made up of a plurality of screens indicating the movement can begenerated from the three-dimensional image information.

Moving objects include men, animals, automobiles, etc. The modes ofmovement of men and animals are walking, running, etc. A motioncorresponding to the mode of movement is defined and the moving motionis automatically drawn. In this way, the labor can be saved.

An unrealistic image of a part such as an entertainer jumping over awide river can also be freely produced. The entertainer is not requiredto make a performance, and the binding time is short. Thus, theproduction cost is also reduced.

In the presence of a background, it is sufficient to take pictures froma plurality of angles, thereby making it possible to reduce the cost offabricating large props and location.

According to a preferred embodiment of the invention, the moving imageinformation producing function of the processing device includes afunction of duplicating the scene frame with a small image attachedthereon, a function of moving the small image in the duplicated sceneframe internally of the scene frame, a function of designating the modeof the particular movement of the small image, a function of designatingthe motion time from the small image before duplication to the smallimage after duplication in the scene frame, and a function ofcontrolling the production of the temporal series of moving imageinformation associating with the motion change from the small imagebefore duplication to the small image after duplication within the sceneframe in such a manner as to produce the moving image information forthe designated motion time.

According to this embodiment, after duplicating the scene frame with thesmall image attached thereon, the small image is moved within theparticular duplicated scene frame.

Now, the information on the transition from the position of the smallimage in the original scene frame to the position thereof in theduplicated scene frame can be obtained. Also, the mode of movement formoving the small image is designated.

The mode of movement is defined as the form of motion such as “walk”,“run”, “fly”, etc. The motion time from the small image beforeduplication to the small image after duplication is designated as asomewhat long time in the case where a relatively slow movement isdesired and as a somewhat short time in the case where a relativelyrapid movement is desired.

Upon preparation as described above, a temporal series of moving imageinformation for motion change from the small image before duplication tothe small image after duplication is produced. At the same time, themoving image information for the motion time designated in the mode ofmovement described above can be automatically produced.

According to another preferred embodiment of the invention, theprocessing device further includes a function of changing a visual fieldin the three-dimensional space in the duplicated scene frame and afunction of controlling the production of the temporal series of movingimage information relating to the motion change from the small imagebefore duplication to the small image after duplication in such a manneras to produce the moving image information with the visual fieldchanged.

This function makes it possible not only to produce the temporal seriesof moving image information with a single fixed visual field but also toproduce dynamic moving image information with a movable visual field.Specifically, an moving image can be produced in a zoomed fashion.

According to still another preferred embodiment of the invention, theprocessing device further includes a function of changing a view pointin the three-dimensional space in the duplicated scene frame and afunction of controlling the production of a temporal series of themoving image information relating to the motion change from the smallimage before duplication to the small image after duplication with theview point changed.

The operation of this embodiment is as follows. The temporal series ofmoving image information can be produced either with the view pointfixed at a point or dynamically by moving the view point. In otherwords, it is possible to produce the moving image information with thecamera moved or the visual direction changed (panned).

According to yet another preferred embodiment of the invention, theprocessing device further includes a function of displaying a scenedescription frame for describing a scene of a scene frame in associationwith the particular scene frame, a function of displaying a speech framefor describing speech in the scene of the scene frame, and a function ofattaching the speech described in the speech frame to a small imageselected in the scene frame.

The operation of this embodiment is as follows. The outline of theprocedure for producing a moving image program (movie, TV program, etc.)includes original work, script, performance/imaging, editing andpost-production. The original work is an ordinary book written by anauthor. An original work which is either superior or has a feature isselected to plan the production of a moving image program.

Next, a script is prepared. The script is a description of the contentsof the moving image program. Performers, scene construction, speech,etc. are described in detail. The script is configured mainly of ascene, scene description and speech of the performers as a combination(which is called a storyboard).

Generally, the storyboard is often prepared by a plurality of producersfor each scene. A storyboard production tool is equipped with a functionof combining or replacing a plurality of storyboards in a designatedorder for each scene. This function corresponds to editing work forprogram production.

In conventional program production, all scenes are imaged with theperformers giving a performance.

According to this invention, in contrast, the three-dimensional imageinformation of the performers is prepared on a processing device such asa computer, and the performance is created on the computer, therebymaking it possible to produce a moving image. The speech can also beproduced on the computer.

This invention is intended to produce storyboards or scripts easilyusing the computer. Also, the invention is intended to produce a movingimage by describing the scene construction of storyboards in detail anddesignating the movement information between the storyboards.

In a scene description frame, a description is made as to what scene isinvolved in the corresponding scene frame. In a speech frame, on theother hand, the speech spoken in the corresponding scene frame isdescribed. In order to associate the speech with a small image, thespeech in the speech frame is attached on the small image in the sceneframe.

The production of a storyboard is made possible by a low-speed computersuch as a personal computer using the simple image information of apart. A storyboard is produced on the personal computer using thefollowing tools.

The personal computer is supplied with the simple image information ofthe part required for producing a storyboard. A scene frame forindicating a scene, a small image frame for displaying a plurality ofinput small images, a scene description frame for describing the sceneand a speech frame for inputting speech are displayed on a screen of thepersonal computer.

A symbol indicating a view point is displayed in the scene frame therebyto designate a standard visual field. A small-screen is selected and asmall image is attached at an appropriate position in the scene frame bythe drag-and-drop operation or the like.

By operating the mouse, the size and direction of the part are set. Ascene is described by attaching a plurality of parts. The informationincluding a program title, a scene name and a scene number is input inthe scene description frame.

Speech is described in a speech column. For the speech, a speaker andtime from the start of a particular scene to the start of the speech aredesignated. Also, if necessary, voices are automatically generated and amouth of the corresponding part is moved in the same manner as if thespeech is generated. Thus, one complete storyboard is produced.

As described above, a storyboard with a scene frame, a scene descriptionframe and a speech frame associated with each other can be easilyproduced using the low-speed image information production device such asa personal computer making up a low-speed computer.

According to a further preferred embodiment of the invention, thecomputer further has a function of designating the timing of generatingthe speech described in the speech frame and a function of synthesizingand outputting voices of the corresponding speech at the designatedtiming.

The operation of this embodiment is as follows. With regard to thespeech described in the storyboard, a small image of a speaker or thelike is designated and associated with speech, while at the same timedesignating the timing, i.e. the period of time from the start of theparticular scene to the start of the speech.

As required, the pronunciation mode of the voice of the speaker asassociated with the small image or the order of making the speech isdesignated, and so is the time from the end of the preceding speech tothe start of the current speech.

The voice of the corresponding speech is synthesized at the designatedtiming. In this way, the voice information for giving a speech isproduced in a manner designated for a particular person while at thesame time producing a moving image. In other words, the speech can beautomatically generated from the storyboard.

Nevertheless, the voice information is not necessarily automaticallygenerated as described above but may alternatively be produced by amethod (dubbing) in which a voice entertainer speaks in accordance withthe images as in the prior art. Also in this case, the cost is not sohigh and poses no special problem.

According to a still further preferred embodiment of the invention, theprocessing device further includes a control function for moving a smallimage indicating a mouth in the small image in the pronunciation form insynchronism with the output timing of the speech.

The operation of this embodiment is as follows. At the same time that amoving image is produced, the motion of automatically moving the mouthof a particular person in the same manner as if the speech described inthe storyboard is pronounced is added automatically. Thus, an additionaljob for moving the mouth is eliminated.

In the way described above, a storyboard is completed.

According to a yet further preferred embodiment of the invention, theprocessing device further includes a function of collectively managing aplurality of storyboards each constituting the information describingone scene configured of a scene frame, a scene description frame and aspeech frame.

A collection of a plurality of storyboards can be called a script. Aplurality of storyboards corresponding to one program, for example, maybe called a script or otherwise the script can be managed as a piece ofeffective video information.

According to another preferred embodiment of the invention, theprocessing device further includes a function of designating one of aplurality of the above-mentioned managed scripts describing a unitbroadcast program configured of a plurality of storyboards and upondesignation of a specific storyboard, displaying a moving image from theparticular storyboard or in the absence of designation, displayingmoving image information each from the beginning, and a function ofoutputting the voice information as required.

The operation of this embodiment is as follows. A script or a storyboardconstituting a part of the script is designated, and a moving imagedescribed in the storyboard is generated and displayed or, if required,voices are generated and output from the particular part. Thus, thescript can be previewed (confirmed by being reproduced).

For example, a moving image produced in compliance with a previewcommand can be viewed on the screen of the personal computer. Thismoving image is normally based on the simple image information. Thoughit has a rough screen, the contents of the moving image can beconfirmed.

In any case of inconvenience, each scene is corrected, and the processof generating and previewing a moving image is repeated to obtain acomplete one. In the case where the scene changes, a new scene isdesignated. Then, both of the scene frame and the scene name are erased,and a new scene can be input.

According to still another preferred embodiment of the invention, thereis provided a video information producing device, wherein the processingdevice is further configured so that a high-speed image informationproduction device and a low-speed image information production devicecollaborates with each other, the high-speed image informationproduction device having a function of producing simple imageinformation with image information partly deleted from real-imagethree-dimensional image information and transferring the data to thelow-speed image information production device, the low-speed imageinformation production device having a function of executing the processof each function in the simple image information transferred as data inthe simple image information transferred thereto as data, producing ascript based on the simple image information and transferring the datato the high-speed image information production device, the high-speedimage information production device having a function of converting thescript based on the simple image information transferred thereto as datainto a script produced based on the real-image three-dimensional imageinformation as detailed image information.

The operation of the embodiment will be described below. The high-speedimage information production device such as a high-speed computerproduces the simple image information, i.e. the information with imageinformation partly deleted from the real-image three-dimensional imageinformation indicating a person or the like or a background, andtransfers the simple image information as data to the low-speed imageinformation production device.

The low-speed image information production device, on the other hand,executes the process of each function described above in the simpleimage information transferred thereto as data, produces the script basedon the simple image information and transfers the particular script asdata to the high-speed computer.

The high-speed image information production device converts the scriptbased on the simple image information transferred thereto as data intothe script produced based on the real-image three-dimensional imageinformation as detailed image information.

More specifically, the high-speed computer inputs the detailed imageinformation of the parts required for producing the storyboard for theparticular script.

The tool for producing the storyboard is operated in the same way on thehigh-speed computer. In response to a moving image production command,however, a moving image is produced according to the detailed imageinformation of the parts.

The high-speed computer has no function of preview on the CRT screen buta function of display on a screen or a monitor TV for confirmation inthe real environment.

In the presence of any inconvenience, the storyboard on the high-speedcomputer is corrected. For sad expression of an entertainer or othercase requiring a delicate expression, the performance of the entertaineris actually imaged and input as a part (moving part) and attached on thestoryboard. In this way, a real image can be also inserted.

As described above, the script prepared using the simple imageinformation in a low-speed system such as a personal computer is inputto a high-speed system, so that the moving image information and thevoice information of the script can be produced using the originalthree-dimensional image information.

Incidentally, the parts information can be used also for other programs,and an environment must also be constructed in which the partsinformation are stored in a large-capacity storage device andcollectively managed for reuse. The reuse of the parts information isexpected to reduce the program production cost further.

With the future improvement of the performance of the personal computer,storyboards may become able to be produced also by the personal computerbased on the detailed image information.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram for explaining an example of a high-speed computersystem configuration of a video information producing device accordingto a best mode for carrying out the invention.

FIG. 2 is a diagram for explaining an example of a system configurationwith a personal computer based on a video information producing deviceaccording to a best mode for carrying out the invention.

FIG. 3 is a diagram for explaining an example of a no-contactthree-dimensional digitizer of a video information producing deviceaccording to a best mode for carrying out the invention.

FIG. 4 is a diagram for explaining the scene production as an example ofoperation of a video information producing device according to a bestmode for carrying out the invention.

FIG. 5 is a diagram for explaining the scene duplication and themovement of persons as an example of operation of a video informationproducing device according to a best mode for carrying out theinvention.

FIG. 6 is a diagram for explaining the scene duplication and themovement of a viewpoint as an example of operation of a videoinformation producing device according to the best mode for carrying outthe invention.

FIG. 7 is a diagram for explaining the form of motion and movement as anexample of operation of a video information producing device accordingto a best mode for carrying out the invention.

FIG. 8 is a diagram for explaining a skeleton as an example of operationof a video information producing device according to a best mode forcarrying out the invention.

FIG. 9 is a diagram for explaining a storyboard as an example ofoperation of a video information producing device according to the bestmode for carrying out the invention.

FIG. 10 is a diagram for explaining the attachment of speech as anexample of operation of a video information producing device accordingto a best mode for carrying out the invention.

FIG. 11 is a diagram for explaining an example of functions of ahigh-speed computer and a personal computer in a video informationproducing device according to a best mode for carrying out theinvention.

BEST MODE FOR CARRYING OUT THE INVENTION

A video information producing device according to a best mode forcarrying out the invention will be explained in detail below withreference to the drawings.

Referring to FIG. 1, a configuration of a high-speed computer system ofthe video information producing device will be explained. A digitizer 1is a three-dimensional image information input device for inputting thethree-dimensional image information from an actual object or aphotograph.

A high-speed computer body 2 has a single or a plurality of high-speedprocessors and is a computer having a large-capacity storage function.

A mouse 3 and a keyboard 5 as operating devices are for operating thehigh-speed computer body 2.

A CRT 4 making up a display device is for displaying the information ofthe high-speed computer body 2. A monitor TV 6 making up a displaydevice, on the other hand, is for displaying the image output from thehigh-speed computer body 2.

The high-speed computer body 2 is included in a processing device. Inthis case, the mouse 3, the keyboard 5, the CRT 4 and the monitor TV 6can also be included in the processing device.

The component parts making up the processing device may be configured aseither an integral unit or separate units, and may transmit/receiveinformation to and from each other by cable or radio.

A CG input function 7 indicates that the high-speed computer body 2 hasa function of producing an image and capturing the image from thedigitizer 1.

A script producing function 8 indicates that the high-speed computerbody 2 has a function of inputting/outputting a script configured of ascene frame, a scene description frame and a speech frame.

A moving image producing function 9 indicates that the high-speedcomputer body 2 has a function of producing a moving image as designatedby the script and a function of visual check (preview).

A simple image producing function and a small image producing function10 indicate that the high-speed computer body 2 has a function ofproducing a small image used for preparing a script and a function ofproducing a simple image used for preview.

A simple image/small image input/output function 10 indicates that thehigh-speed computer body 2 has a function of input/output with othercomputers through a network communication function 11.

The network communication function 11 indicates the availability of afunction of communication with other high-speed computers or personalcomputers.

With reference to FIG. 2, a system configuration of a personal computerof the video information producing device will be explained. A digitizer12 is a three-dimensional image information input device for inputtingthree-dimensional image information from an actual object or photograph(real image).

The digitizer 12 is defined by dashed line to indicate that thisfunction is executed normally by the high-speed computer shown in FIG. 1but not by a personal computer system.

A CRT 13 making up a display device displays the display informationfrom a personal computer body 14. The personal computer body 14 hasbuilt therein a processor and a storage unit. A keyboard 15 and a mouse16 making up operating devices are for operating the personal computerbody 14.

In addition to the personal computer body 14 which is included in theprocessing device, the mouse 16, the keyboard 15 and the CRT 13 can alsobe included in the processing device. These component parts making upthe processing device may be configured either integrally or asindependent members, and can transmit/receive information between themeither by cable or by radio.

A CG input function 17 indicates that the personal computer body 14 hasa function of producing a three-dimensional CG image or inputtingthree-dimensional image information by means of the digitizer 12 from anobject or a photograph.

The digitizer 12 is defined by dashed line for indicating that thisfunction is normally executed by the high-speed computer shown in FIG. 1and used by inputting information by means of a simple image/small imageinput/output function 20.

A script producing function 18 indicates that the personal computer body14 has a function of inputting/outputting a script configured of a sceneframe, a scene description frame and a speech frame.

A moving image producing function 19 indicates that the personalcomputer body 14 has a function of producing and previewing a movingimage based on a simple image.

The simple image/small image input/output function 20 indicates that thepersonal computer body 14 has the function of inputting the simpleimage/small image produced by the high-speed computer shown in FIG. 1 ora function of outputting the simple image/small image stored.

A network communication function 21 indicates that the personal computer14 has a function of communicating with other personal computers orhigh-speed computers.

FIG. 3 shows an example of a no-contact three-dimensional digitizer. Anobject 23 is imaged simultaneously by five cameras including a firstcamera 24, a second camera 25, a third camera 26, a fourth camera 27 anda fifth camera 28. This information is calculated thereby to produce thethree-dimensional image information of the object 23.

The steps of the object are calculated from the images taken from twodifferent directions. This process is executed by all of adjacentcameras. In the drawing, the mechanism for capturing the image of eachcamera into the high-speed computer body 2 is not shown.

With reference to FIG. 4, the scene production will be explained as anexample of the operation of the video information producing device. Asmall image frame 71 displays a small image representing thethree-dimensional image information. A small image 72 shows a house, asmall image 73 a person A, a small image 74 a tree, a small image 75 asignboard and a small image 76 a person B.

A scene frame 61 indicates a scene. Trees 62 are formed of the smallimages 74 attached by the drag-and-drop operation.

In the process, the small image 74 remains as it is in the small imageframe 71, and duplicated small images are attached. A house 63 is thesmall image 72 attached, a person A 64 the small image 76 attached, theperson B 65 the small image 73 attached, and a signboard 66 the smallimage 75 attached.

The small images are attached in the scene frame 61 in such a manner asto involve individual images with a proper noun or the like to identifyeach person on the scene rather than entities of three-dimensional imageinformation such as the persons A and B.

For fine adjustment of the direction, size and position of the smallimages attached, each small image is double-clicked to displaythree-dimensional image information based on the simple imageinformation at the position of the particular small image. At the sametime, a pull-down menu or the like is displayed nearby. In this way, thefine adjustment of size, rotation, distance and lateral direction ismade possible. At the end of the pull-down menu (example), the smallimage display is restored from the three-dimensional real image display.

With reference to FIG. 5, an example of the duplication of a scene and amoving person will be explained as an example of operation of the videoinformation producing device. A scene frame 81 is a direct duplicationof the scene frame 61 with small images of persons and the like attachedthereon. Small images 82, 83, 86 are the very duplications.

The small image 64 is moved to the small image 84, and the small image65 to the small image 85. The small images defined by dashed linesindicate the positions before movement. For detailed setting of thesmall images 84, 85 after movement, the real image display by doubleclick and the fine adjustment with a pull-down menu are used. In thiscase, the movement is involved, and therefore the items of mode ofmovement are also displayed in the pull-down menu, so that the mode ofmovement such as “walk” or “run” is also set.

By setting the time between the scene frame 61 and the scene frame 81,the moving image between the scene frame 61 and the scene frame 81 isproduced.

FIG. 6 shows an example of a duplication of a scene and the movement ofa viewpoint. In FIG. 6, a scene frame 91 is a duplication of the sceneframe 81 shown in FIG. 5.

Small images 92, 93, 94, 95, 96 of the scene frame 91 are not moved. Acamera position 97 is located at the center of the scene. The cameraposition 97 is moved to a position 98 by the drag-and-drop operation.

Once the scene frame 91 is duplicated, a duplication is completed suchas a scene frame 99 by the movement of the camera position. As long as amoving image is produced by designating the time between the scene frame91 and the scene frame 99, a moving image is produced with the movementof the camera position.

Although only the camera position is moved between the scene frame 91and the scene frame 99, the visual field (zoom) of the camera can alsobe changed as well as the person and the camera position.

FIG. 7 is a diagram for explaining the form of motion and movement. InFIG. 7, numeral 31 designates a view of a person in standing posturetaken from the left side. A person 32 has both of his left hand and hisright foot placed slightly forward with this left foot in touch with theground. The foot in touch with the ground is fixed on the ground.

A person 33 is a slightly more advanced in motion, while a person 34 hashis right foot in touch with the ground. The persons 31 to 34 representa case in which a person walks. By carrying out this motion on left andright sides alternately, a walking motion is realized.

FIG. 8 shows a skeleton of a person. The skeleton is configured ofjoints as supporting points, which are connected by bones.

Joints 41, 42 are those of the shoulder and remain comparativelystationary when walking. A joint 43 is that of an arm and is connectedwith the joint 41 through a bone. Therefore, the distance between thejoints 41 and 43 remains unchanged. A joint 44 is that of the right art.Similarly, joints 45 to 50 are those of the body.

The motion of a person is implemented by the movement of movable joints.In order to realize a complex motion and a more natural motion of aperson in terms of CG, a person wearing an instrument for measuring thejoint positions is moved to acquire the position information of thejoints. This job is called motion capture, whereby the natural motion ofan animal can be realized as well as that of the person.

The mode of movement is often a standard one. According to thisinvention, the standard motion is predetermined by acquiring the motioninformation by means of the motion capture or the like. In the casewhere a mode of movement is designated in the pull-down menu, the motionis implemented based on the motion information corresponding to theparticular mode of movement.

For some programs, however, an expression with the mode of movementprepared as a standard motion is impossible, and the motion capture isrequired in many scenes. As usable modes of the motion capture, the modeof movement such as stealthy footsteps or shy motion is added to thepull-down menu.

FIG. 9 shows an example of a storyboard. In FIG. 9, a scene frame 101has small images attached. Speech 102 includes those of the persons Aand B described. A scene description frame 103 contains the descriptionof the scene number, the clip number, etc. The scene frame 101, thespeech 102 and the scene description frame 103 make up one storyboard.

FIG. 10 is a diagram for explaining the attachment of speech. Aattaching job 111 is carried out in such a manner that speech A isattached on a person on the scene by the drag-and-drop operation of themouse, while a attaching job 112 is implemented by attaching speech B onanother person on the scene.

At the time of attachment, a panel 113 is displayed nearby, so that thespeech starting time from the initiation of the scene and the time ofintermission before the next speech after the end of the precedingspeech can be designated. The upper right panel 113 is an enlarged viewof the panel displayed.

The attaching jobs enable a person to announce the speech. The personcan be set either to move his/her mouth only or to issue voices by voicesynthesis.

In the state of art, however, the utterance by voice synthesis fails toachieve the natural voice of the man but is limited, for the time being,to the voice of a robot or an animal other than a man. The utterance byman is produced by a (after-recording) technique in which the speech areissued and recorded while watching the moving image after completion ofthe moving image and matching voices with the moving image later.

In this way, one storyboard is completed. Upon receipt of a command forproducing the next storyboard, the scene number is incremented and thespeech are erased without changing the contents of the scene frame. Thesmall images displayed in the scene frame are moved by the drag-and-dropoperation or the like and the amount of movement of the particular partsis designated. The small images thus moved are double-clicked, so thatthe mode of movement such as “walk” or “run” is designated by thepull-down menu.

For producing a moving image (composed of a plurality of images)involving a motion for movement, the definition of the skeleton ofparts, the motion capture, the correction of slide or the likesophisticated techniques are required for generating a natural motion.This technique, however, is not the gist of the invention.

With reference to FIG. 11, the functions of the high-speed computer andthe personal computer will be explained. A flowchart 121 represents theprocedure for producing a program by a high-speed computer. Thehigh-speed computer corresponds to a high-speed image informationproduction device and the personal computer to a low-speed imageinformation production device.

The three-dimensional CG information input 122 is the job for inputtingthe CG information by use of the keyboard 5 and the mouse 3. Theinformation thus input are stored as three-dimensional imageinformation.

Digitizer real image information input 123 is the job for inputtingthree-dimensional image information from a real object using ano-contact three-dimensional digitizer or the like shown in FIG. 3. Theinformation thus input is stored as three-dimensional image information.

Image information simplification 124 is a job for simplifying the storedthree-dimensional image information and storing it as simple imageinformation thereby to produce and store a small image.

Storyboard production 125 is a job for constructing a scene by attachingsmall images on a scene frame, duplicating the scene, designating themovement and the mode of movement of small images, describing a sceneand speech and attaching the speech.

The three-dimensional image information used in this case may be eitherthe simple image information or the detailed image information. This jobframe is indicated by dashed line, which indicates the job is normallyperformed by the personal computer.

Script check 126 is a job for producing, viewing and confirming themoving image by the detailed image information making up non-simpleimage information. Moving image production 127 is a job for producing amoving image based on the detailed image information from the scriptconfirmed.

A flowchart 131 shows the procedure for producing a program by apersonal computer.

Three-dimensional CG information input 132 is a job for inputting the CGinformation by use of the keyboard 15 and the mouse 16. The informationthus input are stored as three-dimensional image information.

Digitizer real image information input 133 is a job for inputtingthree-dimensional image information from a real object using ano-contact three-dimensional digitizer or the like shown in FIG. 3. Theinformation thus input is stored as three-dimensional image information.These jobs are carried out normally by a high-speed computer.

Image information simplification 134 is a job for simplifying the storedthree-dimensional image information, storing it as simple imageinformation, and producing and storing a small image. This job is alsocarried out normally by a high-speed computer.

Storyboard production 135 is a job for configuring a scene by attachingsmall images on a scene frame, designating a scene duplication and themovement and the mode of movement of small images, explaining the sceneand describing the speech and attaching the speech. This job is carriedout on the personal computer using the simple image information.

Script check 136 is a job for producing the motion from the script,viewing and checking it using the simple image information.

Script check 137 is a job for producing a moving image and viewing andchecking it from the script using the detailed image information. Thisjob is normally carried out by a high-speed computer but mayalternatively be executed by a personal computer at the sacrifice ofconsiderable time consumption.

Moving image production 138 is a job for producing a moving image fromthe script using the detailed image information. This job is normallycarried out by a high-speed computer.

The arrow from the image information simplification 124 to thestoryboard production 135 indicates that the simple image informationproduced by the high-speed computer is transferred by the personalcomputer and the storyboard is produced on the personal computer.

The arrow from the script check 136 to the script check 126 indicatesthat the script produced by the personal computer is transferred to thehigh-speed computer and a moving image is produced with the detailedimage information.

With the video information producing device described above, a tool forproducing the script and the storyboard on the computer is produced andby combining the storyboard with the existing CG technique and the voicesynthesis technique, the effects of the following items can be realized.

1. The three-dimensional image information of an existing object orperson is captured, and a moving image of the existing object or personcan be produced using the CG technique. Thus, the time binding anentertainer can be remarkably reduced while at the same timeconsiderably reducing the program production cost by eliminating theneed of large props utilizing natural landscape.

2. The use of the CG technique makes it possible to freely produce amoving image of a dangerous motion of an existing entertainer such asfalling off from a high place or an unrealistic motion such as jumpingover a wide river in one breath.

3. A program can be produced on the desk by producing a script and astoryboard on the computer and producing and previewing a moving imagefrom the storyboard. As a result, the program production cost can bereduced.

4. By producing the simple image information of three-dimensional imageinformation, a program can be produced with a low-speed computer like apersonal computer. Thus, the script completed on the personal computeris transferred to the high-speed computer, so that a moving image can beproduced with the detailed image information on the high-speed computer.

5. A moving image of moving a mouth by speech as well as the motion ofmovement can be produced.

6. Voices can be synthesized with the speech.

With the video information producing device described above,three-dimensional image information of an existing object, an existingperson, etc. are captured and a moving image of the existing object orperson can be produced using the computer graphics technique. As aresult, the time binding an entertainer can be considerably reduced.Also, the use of three-dimensional image information of an existingobject as a background eliminates the large props, or otherwise reducesthe program product cost remarkably.

Also, based on the real-image three-dimensional image information of anentertainer, the moving image information can be produced in a desiredsituation within the scope of the computer graphics technique. As aresult, dangerous motions such as jumping down from a high place or anunexpected, unrealistic motions such as jumping over a wide river in abreath can be produced freely as a moving image.

Further, simple image information is produced by a high-speed imageinformation production device and delivered to a low-speed imageinformation production device. In this way, storyboards and a script areproduced by the low-speed image information production device, and basedon this script, the high-speed image information production deviceproduces a moving image with the detailed image information. Thus, themain work can be carried out on a low-speed image information productiondevice such as a personal computer. As a result, a moving image of anexisting part can be produced both easily and efficiently at low cost.

INDUSTRIAL APPLICABILITY

This invention is applicable to the video information producing devicefor producing a moving image by the computer graphics technique.

1. A video information producing device comprising: a processor; anoperating device for operating the processor; and a display device fordisplaying information of the processor; wherein the processor performsa function of storing real-image three-dimensional image information, afunction of generating small image information from the real-imagethree-dimensional image information read from the storage function, afunction of storing the real-image three-dimensional image informationand the small image information as associated with each other, afunction of displaying a scene frame for expression of three-dimensionalspace, a function of displaying a small image of the small imageinformation in a small image frame, a function of selecting the smallimage in the small image frame on a screen and moving and attaching thesmall image at a desired position in the scene frame, a function ofstoring information for arrangement of the attached small image in thescene frame, a function of correcting the form of expression of thesmall image displayed in the scene frame internally of the scene frame,a function of changing the form of motions of the small image displayedin the scene frame internally of the scene frame, a function of updatingthe arrangement information of the small image after the correction orafter the change, a function of generating a temporal series of movingimage information by calculating the motion change from the small imagebefore the change to the small image after the change in the sceneframe, and a function of storing the generated moving image information.2. A video information producing device comprising: an input device forinputting real-image three-dimensional image information; and aprocessing device for processing the three-dimensional image informationfrom the input device; wherein the processing device performs a functionof storing the real-image three-dimensional image information, afunction of generating small image information from the real-imagethree-dimensional image information read from the storage function, afunction of storing the real-image three-dimensional image informationand the small image information as associated with each other, afunction of displaying a scene frame for expression of three-dimensionalspace, a function of displaying a small image of the small imageinformation internally of the small image frame, a function of selectingthe small image in the small image frame on a screen and moving andattaching the small image at a desired position in the scene frame, afunction of storing information of arrangement of the attached smallimage in the scene frame, a function of correcting the form ofexpression of the small image displayed in the scene frame internally ofthe scene frame, a function of changing the form of motions of the smallimage displayed in the scene frame internally of the scene frame, afunction of updating the arrangement information of the small imageafter the correction or after the change, a function of generating atemporal series of moving image information by calculating the motionchange from the small image before the change to the small image afterthe change in the scene frame, and a function of storing the generatedmoving image information.
 3. A video information producing deviceaccording to claim 2, wherein the moving image information generatingfunction of the processing device includes functions of; duplicating thescene frame with the small image attached thereto, moving the smallimage in the duplicated scene frame internally of the scene frame,designating movement mode of the small image, designating motion time ofmotion from the small image before duplication to the small image afterduplication in the scene frame, and controlling generation of a temporalseries of the moving image information for the motion change from thesmall image before duplication to the small image after duplication insuch a manner as to generate the moving image information for thedesignated motion time.
 4. A video information producing deviceaccording to claim 3, wherein the processing device further includes afunction of changing a visual field in the three-dimensional space ofthe duplicated scene frame and a function of controlling generation of atemporal series of the moving image information for the motion changefrom the small image before duplication to the small image afterduplication in such a manner as to generate the moving image informationwith the change of the visual field added thereto.
 5. A videoinformation producing device according to claim 2, wherein theprocessing device further includes a function of changing a visual fieldin the three-dimensional space of the duplicated scene frame and afunction of controlling generation of a temporal series of the movingimage information for the motion change from the small image beforeduplication to the small image after duplication in such a manner as togenerate the moving image information with the change of the visualfield added thereto.
 6. A video information producing device accordingto claim 2, wherein the processing device further includes a function ofdisplaying a scene description frame for describing a scene of the sceneframe together with the scene frame, a function of displaying a speechframe for describing a speech in the scene of the scene frame, and afunction of attaching the speech described in the speech frame to asmall image selected in the scene frame.
 7. A video informationproducing device according to claim 6, wherein the processing devicefurther includes a function of designating timing of generating thespeech described in the speech frame and a function of synthesizing andoutputting a voice of the corresponding speech at the designated timing.8. A video information producing device according to claim 7, whereinthe processing device further includes a function of controlling thesmall image indicating a mouth for pronouncing movement in synchronismwith the output timing of the speech.
 9. A video information producingdevice according to claim 6, wherein the processing device furtherincludes a function of collectively managing a plurality of storyboardseach making up a unit storyboard including the information fordescribing one scene configured of a scene frame, a scene descriptionframe and a speech frame.
 10. A video information producing deviceaccording to claim 9, wherein the processing device further includes afunction of designating one of a plurality of the managed scripts eachincluding a plurality of storyboards and description of a unit broadcastprogram and upon designation of a specific storyboard, displaying themoving image information from the storyboard and unless so designated,displaying the moving image information each from the beginning, and afunction of outputting the voice information as required.
 11. A videoinformation producing device according to claim 2, wherein theprocessing device is so configured that a high-speed image informationproduction device and a low-speed image information production devicecollaborate with each other, wherein the high-speed image informationproduction device has a function of producing simple image informationwith a part of the image information deleted from the real-imagethree-dimensional image information and transferring the data to thelow-speed image information production device, wherein the low-speedimage information production device has a function of executing theprocess of each function for the simple image information transferredthereto as data, producing a script based on the simple imageinformation and transferring the data to the high-speed imageinformation production device, and wherein the high-speed imageinformation production device has a function of converting the scriptbased on the simple image information transferred thereto as data into ascript based on the real-image three-dimensional image informationconstituting the detailed image information.
 12. A video informationproducing device according to claim 2, wherein the processing deviceincludes a computer body having a function of storing and processinginformation, an operating device for operating the computer body and adisplay device for displaying information from the computer body, andwherein the computer body executes each of the functions in accordancewith the operation of the operating device and causes requiredinformation to be displayed on the display device.